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Computer-Assisted System for Designing Training Programs 

Field of the Invention 

The invention relates to a computer-assisted 
system enabling the training/learning function of an 
enterprise to produce training/learning programs, using 
off-the-shelf and bespoke training/ learning products and 
5 services. It is particularly concerned with a computer- 
assisted system for assisting the production, planning, 
control, tracking and other management aspects of 
training/learning programs for individuals or groups of 
individuals of an enterprise, with a view to the 
10 individuals (s) achieving learning outcomes by implementing 
learning objects. 

Background of the Invention 

For convenience, various terms used herein are 
defined as follows: 

15 "Learning Objects" (LO or LO's): courses, tests, 

exercises, simulations, classroom sessions; tutorial 
sessions, manuals, games, props etc., all in electronic, 
paper or material form. 

"Description Model": an example of a standard 
20 description of a LO. 

"Outcomes": the learning program and LO objectives 
expressed in term of what students can do after having 
taken the learning program/LO. 

"Outcomes Thesaurus": a set of normalised outcome 
25 names, synonyms and descriptions. 

"Parameters": elements which define a 
training/learning program or LO, e.g. cost, duration, 
number of students, schedule, location, facilities, type 
of instruction, etc. 

30 "Parameter Model": an example of a standard set of 

Enterprise/LO parameters. 
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"Learning Curriculum": an organized list of 
learning activities put together for a target audience to 
reach a set of capabilities that will support 
business /enterprise outcomes. In other words, a curriculum 
5 is a set of learning objects that are necessary to bring 
set of outcomes to a student. Normally a curriculum does 
not include two LO's that are equivalent, i.e. that are 
associated to the same set of outcomes . 

"Pre-requisite" is a set of outcomes that must be 
10 acquired by a student before he takes a given LO. The set 
of LO's in a curriculum is not necessarily a set linked 
through pre-requisites . 

"Enterprise": commercial undertaking or other 
organization that employs persons to carry out tasks. 

15 "Students": an enterprise's staff, partners and/or 

customers . 

When an enterprise's training department organizes 
a training program, it is usually in response to an 
explicit or implicit requirement/need of the organization. 

20 The inputs that will be taken into consideration are 
basically a description of expected outcomes, a target 
audience, and constraints. The expected outcomes can be 
given in terms of business requirements, which will have 
to be translated in capability requirements and 

25 identification of a target audience, or directly in 
capability requirements. The target audience are the 
people involved in achieving the above business 
requirements. The constraints include time, budget and 
other constraints, e.g; environmental, infrastructure, 

30 policy, practice, etc. 

Based on the above inputs, the training/program 
manager will have to define the best ways to achieve the 
acquisition of the capabilities necessary to achieve the 
work required to reach the expected outcomes, within the 
35 given time, budgetary and other constraints. 

The required capabilities translate into the 
potential learning objectives for the target audience. 
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Each learning objective can be reached by a learning 
activity, or "Learning Object" (LO) , or a set thereof. 

Organized lists of learning activities each put 
together for a target audience to reach a set of 
5 capabilities that will support business outcomes 
constitute a set of learning curricula, and a training 
program will use the curricula within a set of managed 
processes, where time and other constraints are taken into 
account . 

10 The reason of being of a training program is to 

increase workers efficiency at work. When an organization 
launches a training program, it is to support specific 
enterprise objectives, which are expressed in terms of 
desired outcomes. The purpose of this step is to define 

15 the end goal, which will also be used to apply 
measurements of success. 

The definition of the business objectives is 
usually given as an input to the training department, as 
well as the target audience. 

20 For the target audience, in organizational terms 

it makes sense to first look for people having the closest 
possible capabilities already available, and see how to 
bring them together to achieve the bespoke business 
objectives, before defining training curricula and 

25 programs. But this decision is an organizational decision, 
not a training department decision. It is assumed that 
people capabilities and availability criteria are not met 
in the target audience, which is why the training program 
has got justification. 

30 The desired business outcomes are often specified 

in the training program request. If they are - and even 
more if they are not - it is necessary for the training 
program manager to make sure that these objectives are 
clear, and to reformulate them in normalised terms, so 

35 that they can be translated into "desired workforce 
capability outcomes". This will lead to the next phase, 
defining and prioritizing the set of learning objectives 
that will helo define the content of the Droaram. 
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A curriculum is composed of a set of LO's and a 
set of outcomes; more generally, however, a curriculum can 
be built from LO's only, outcomes only or selection of 
sets of LO ' s and outcomes . 

5 The input for building a curriculum is a set of 

outcomes and/or a list of LO. The output is an ordered set 
of LO's (=curriculum) with associated outcomes and an 
indication, if applicable, of the "coverage" . 

To complete a training project within a training 
10 program, the training manager needs to attach students and 
parameters to a curriculum. First the curriculum manager 
selects one or more than one group of students and 
attaches it to a curriculum. 

To build a curriculum, it is necessary to 
15 understand the starting point, so the learning curriculum 
can be derived from a good representation of current state 
and desired future state. 

A basic curriculum is built on the basis of 
fundamental assumptions about the target audience 

20 capabilities. This first requires an assessment of the 
capabilities the target audience should have . From this, a 
distinction is drawn between the capabilities from the 
"should have" list, which the target audience already has, 
and the capabilities from the "should have" list which the 

25 target audience does not have, and needs to acquire. 

The assumptions can be checked or replaced by a 
Training Needs Analysis. 

As a result, the program outcomes can be defined 
in terms of capabilities to acquire, i.e. learning 
30 objectives. We now know, not only where we want to go (the 
desired future state), but also where we come from (the 
current state) , and get ready to define a road map to get 
there . 

The next step will be the definition of the 
35 learning curricula. 
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Each capability to develop is a potential learning 
objective for the target audience. 

Each learning objective can be reached by a 
learning activity, or "Learning Object" (LO) , or a set 
5 thereof . 

The organized list of learning activities put 
together for a target audience to reach a set of 
capabilities that will support enterprise goals (desired 
outcomes) constitute a learning curriculum. 

10 For better efficiency of the training program, the 

curricula will ideally be customized to the personal 
training needs of the target audience members, as per the 
result of the training needs analysis, or based on 
personal assessment. This can lead to a number of personal 

15 curricula, all based on a starting standard curriculum 
with some additions or subtractions. 

As each learning objective can be reached by a 
learning activity, or a "Learning Object" (LO) , or a set 
thereof, the training program manager needs to identify 
20 the possible "solutions" to a capability acquisition 
problem, within what is available in the various sources 
he has access to. 

The potential LO can be in any format: in-house 
courses, external courses, online courses, instructor-led, 
25 books, tapes, etc. 

The challenge for the curriculum designer will be 
to make a right and coherent choice within a rich learning 
solutions offer, or to find solutions where there are none 
or not many available at a cost that would be supported by 
30 the available budget, and following modalities that will 
support the time constraints, while reaching the necessary 
quality in terms of learning. 

As stated above an organized list of learning 
activities, put together for a target audience, to reach a 
35 set of capabilities that will support business outcomes 
constitute a learning curriculum. 
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Producing a learning curriculum is a recurrent 
process, starting first with a theoretical, standard, 
basic curriculum, that will need to be refined, and 
involves difficulties in particular because all LO do not 
5 have the same relevance: the target audience may not need 
all of them, since the objectives could already be 
acquired; some of the LO could be complementary, or 
prerequisites to others; some would address very critical 
capabilities; and some could be mandatory, some optional 
10 etc... 

All of these difficulties need to be managed. In 
fact, the production of training/ learning programs for 
individuals or groups of individuals of an enterprise is 
highly complex and increasingly work- intensive as the size 
15 of the enterprise increases together with the number of 
individuals/groups having different training needs, taking 
into account also the variety of available learning 
objects and budget, time/timing and other constraints. 

Reduced training costs and improving staff 
20 performance continue to be a high priority for 
enterprises. There is an increasing demand for on-line 
services to support the deployment, management and on- 
going development of skills. There are numerous 
computerized learning management systems on the market. 
25 However, there is a complete lack of integrated systems 
and tools for managing the production of training/ learning 
programs which are "upstream" of the existing learning 
systems . 

Background Art 

30 There have been many proposals to use computer 

systems in a training context . 

US Patent 5,416,694 is concerned with assessing an 
individual's acquisition of specifically identified skills 
and abilities and the relationship of these skills and 
35 abilities to an identified occupation, using a computer- 
based system. 
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US Patent 5,890,149 relates to an employee 
training and coaching system in an prganization, for 
disseminating collective knowledge of a body of corporate 
employees using a computer system. 

5 US Patent 5,904,485 discloses a computer system in 

which a school curriculum is stored for access by students 
at home. Appropriate material for presentation to each 
student during each learning session is selected by an 
intelligent administrator based on the student 1 s learning 
10 profile. 

US Patent 6,157,808 describes a computerized 
employee certification and training system designed to 
support an efficient management of employee development, 
by developing training materials, career paths and 
15 determining an employee's qualifications and performance. 
This patent is concerned principally with controlling 
certifications for an employee to be able to perform a 
given job. 

However, none of the prior patents offers a 
20 solution to the problem of optimizing the design and 
production of a training plan for different target groups 
based on available learning facilities. 

Summary of the Invention 

The invention aims to facilitate the production of 
25 a training plan that fits the needs of any enterprise, 
making use of available learning objects. It provides a 
computer-assisted system for the production of 
training/ learning programs for individuals or groups of 
individuals of an enterprise, with a view to the 
30 individuals (s) achieving learning outcomes by implementing 
learning objects (LO) , which considerably automates 
design/production of the training program and will enable 
a training manager, once the training needs have been 
identified, to produce a plan for implementing the 
35 training objectives. 

The system according to the invention comprises a 
computer database (hereinafter "database") of normalised 
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LO parameters wherein LO's are described in the framework 
of a normalised description; a database of normalised LO 
outcomes wherein LO outcomes are described in the 
framework of a normalised description; and a database of 
5 learning outcome parameters described in the framework of 
a normalised description corresponding to that of the 
normalised LO outcomes. 

The system further comprises a curricula 
generation module or "learning outcome optimization 

10 module" receiving normalised learning outcome parameters 
and normalised LO parameters, selectable into different 
combinations thereof for producing a set of alternate 
learning curricula, each curriculum comprising a set of 
learning objects that are necessary for the individual (s ) 

15 to reach a set of outcomes. 

A curriculum selection module of the system 
receives curricula from the outcome optimization module 
and said normalised LO outcomes, and is arranged to 
provide an interactive indicia of the match to 

20 requirements of any given LO within a learning curriculum 
selectable by the user choosing LO's corresponding to the 
normalised LO outcome (s) and learning outcomes 
corresponding to the learning outcome parameters, to 
assist ranking the curricula for selection of a 

25 training/ learning plan. 

The computer-assisted system according to the 
invention makes use of available computer hardware 
including adequate processing and memory capacity for 
storage of the computer databases and management software. 

30 The system is programmed with available software for 
example Enterprise JavaBeans™ technology powered by an 
Oracle™ database. This technology allows the processes to 
be encapsulated into objects that are flexible, 
extensible, reusable and can evolve as need be. Using this 

35 technology, the main computer system can be stored in a 
server accessible to the users through an Application 
Service Provider (ASP) model via a network, in particular 
the internet. 
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In one embodiment the system according to the 
invention comprises a learning object analysis and 
description standardisation unit, an outcome standard- 
isation unit, an outcome optimization module and a 
5 curriculum selection module. 

The learning object analysis and description 
standardisation unit comprises a database of normalised LO 
parameters wherein LO's are described in the framework of 
a normalised description, and a database of normalised LO 

10 outcomes wherein LO outcomes are described in the 
framework of a normalised description. Means are provided 
for inputting LO parameters and for comparing input LO 
parameters with an LO parameter model, there further being 
means to assist, prompt or effect the conversion of input 

15 LO parameters into normalised LO parameters and normalised 
LO outcomes manually, semi -manually or automatically. 

The outcome standardisation unit comprises an 
outcomes thesaurus database associated with an outcome 
standardisation module having an input for enterprise 

20 outcomes. The outcomes thesaurus database contains a set 
of normalised outcome names, synonyms and descriptions. 
The outcome standardisation module is arranged to compare 
input enterprise outcomes with standardised outcomes from 
the outcomes thesaurus database and is associated with an 

25 outcome verification module, these modules comprising 
means to assist, prompt or effect the conversion of input 
enterprise outcomes into standardised/verified enterprise 
outcome parameters . 

The outcome standardisation unit is arranged to 
30 normalise and verify the input enterprise outcomes as 
follows. Normalisation is needed because the system 
accepts the outcomes as they are given by the enterprise 
in non-standard format. First it matches them with 
normalised outcomes stored in the outcomes thesaurus 
35 database when the enterprise outcome parameters are 
expressed in the same or very similar language to stored 
normalised outcomes. When however the input enterprise 
outcomes are expressed in words that are different to 
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stored normalised outcomes, these words are converted into 
synonyms thereof taken from those stored in the outcomes 
thesaurus, then matched with normalised outcomes stored in 
the outcomes thesaurus. If there is a failure to find a 
5 satisfactory match with normalised outcomes stored in the 
outcomes thesaurus, the module provides for a verification 
to complete the outcome standardisation. 

In the latter case if some outcomes have been 
partly standardised by replacing some words with synonyms 

10 but there is still a mismatch, the outcome standardisation 
unit is arranged to allow a verification that identifies 
the mismatched partly standardised outcome and ensures 
that it is grammatically and lexically correct. If 
correction if necessary, this can be done by manual input. 

15 After verification, the new outcome is then proposed as a 
verified outcome for addition to the outcomes thesaurus, 
such addition preferably being controlled by the user. In 
this way, the outcomes thesaurus is constantly enriched 
with new normalised outcomes. 

20 The outcome optimization module receives 

standardised/verif ied enterprise outcome parameters from 
the outcome verification module and normalised LO 
parameters from the LO unit, selectable manually, semi- 
manually or automatically into different combinations 

25 thereof to produce a set of alternate learning curricula. 

The curriculum selection module receives curricula 
from the outcome optimization module, normalised LO 
outcomes from the LO unit, and normalised enterprise 
parameters that indicate values an enterprise attributes 

30 to the normalised LO outcomes, this module providing an 
interactive indicia of the match to requirements of any 
given LO within the learning curriculum selectable by 
choice of the normalised LO outcome (s) and the verified 
enterprise outcome (s) for ranking the curricula to select 

35 a training/ learning plan. 

The design of the learning plan or program is the 
organization of the curricula within a set of managed 
processes, where target audience, budget, time and the 
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other constraints are taken into account. Through the 
system according to the invention, this program design 
stage is considerably automated. 

Prior art training proposals using computer 
5 systems are based principally on a student-centric model. 
In contrast, the system according to the invention is 
centred on learning objects and their learning outcomes in 
the context of the enterprises who need to organize the 
training. By this management-centric approach, the 

10 invention focusses on the enterprise's training manager, 
surrounded by the curriculum manager, responsible for the 
preparation of an overall training program in line with 
the enterprise's budget and objectives, as well as the 
training administrator responsible for the deployment of 

15 the training program, the student himself who takes the 
training assigned to him, or prepared by him, and the 
content manager, responsible for the identification and 
acquisition of the training content. 

According to one embodiment, the system of the 
20 invention has a Thesaurus of w Business Outcomes", i.e. 
learning objectives, linked to learning activities/ 
learning objects (LO) . 

It will further provide an automatic presentation 
of potential LO's. 

25 After selection of the learning objectives, the 

user will automatically get a corresponding basic 
curriculum. 

At this stage, the curriculum is composed of all 
LO's that cover at least one of the objectives. Some 
30 objectives are covered by more than one LO, some LO cover 
more than one objective. 

The user may then optimize the curriculum: 1) with 
respect of desired outcomes (e.g. to minimize LO 
redundancy); 2) with respect of the cost and time aspects; 
35 and 3) With respect of other parameters such as use of 
existing facilities or team, internal policies, preferred 
learning styles. 
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At the end of the design of a basic curriculum 
("Standard curriculum"), the LO can be categorized in 
mandatory, additional, and optional. 

With the system according to the invention, a 
5 training manager can: 

- Build and customize curricula; 

- Define the size of the target students 
population; 

- Define the project's time constraints; 

10 -Add to the system's LO database the internal 

learning modules with their various parameters, such as 
cost, language, location, calendar, etc.; and 

-Select the complementary services (e.g. 
synchronous or asynchronous electronic tutoring and/or 

15 conferencing) that will be made available to the students. 

The system can automatically provide the user 
with: a roll-out map; a total budget computation, 
including a quotation for the components delivered and 
charged directly by the system provider; and, where 
20 required, a proposal for a corresponding service level 
agreement . 

With the system according to the invention, 
curriculum customization is another instance of curriculum 
design. Authorisation can be given to an end-user to 
25 customize his own curriculum, also on the basis of a 
standard curriculum that has been assigned to him. The 
necessary authorisations can be managed with a workflow 
feature of the system's program. 

Once the set of curricula is designed, the 
30 training program manager needs to make sure that all LO 
are actually available in his environment and take into 
account their delivery conditions. This may need 
contracting with third parties for instructor-led training 
(ILT), development or acquisition of online courses, 
35 customization, implementation and integration in the 
learning environment, etc. 

It is possible for the system provider to offer a 
vast cataloa of existina LO's from manv different 
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suppliers linked to a Thesaurus of business objectives. 
The client/user can also add his own LO to the suppliers 
catalog. If the client/user has profession-specific LO's 
developed himself, he may be offered the choice to share 
5 these solutions. 

Further features of the invention are set out in 
the following description and in the claims. 

Brief Description of the Drawings 

In the accompanying schematic drawings: 

Fig. 1 is a block diagram of an example of a 
system according to the invention. Its right-hand part 
shows the learning object analysis and description 
standardisation unit. Its left-hand part shows the outcome 
standardisation unit, the outcome optimization module and 
the curriculum selection module. 

Fig. 2 is a learning outcomes /learning objects 

matrix. 

Detailed Description of the Invention 

Table I indicates the process steps A-H carried 
20 out in a system according to the invention, with their 
corresponding inputs and outputs. 

Figs. 1-2 show the layout of a system according 
to the invention including modules A to H which 
correspond to the respective process steps A to H in 
25 Table 1. These Figures show the computer system split 
according to its principal functions, also showing its 
interaction with data from Suppliers via a Mapping Unit 
and a Curriculum Office (of the system manager) , and from 
Enterprises (the users) . 
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TABLE I 



Inputs 


Processes 


Outputs 


• Learning Objects (LO) 

• Description Model 


A. LO Analysis 


• Elements of 
normal i sea iaj 
description 


• Elements of 
normalised LO 
description 

• LO supplier 

QcbUIipUlOllb 

• Outcomes Thesaurus 

rrdJ. GLlllC l~wJL HUUwX 


B. LO Description 
xviorma 1 1 z a u i on 


• Normalised LO 
outcomes 

• Normalised LO 
parameters 


• Enterprise outcomes 

• Outcomes thesaurus 


C. Enterprise 
Outcome 

S t andar di s a t i on 


• Normalised 
enterprise 
outcomes 


• iNJormaiiseQ enterprise 
outcomes (desired) 


u . uuccome 

Verification 


• vcritiea 
Enterprise 
outcomes 


• Enterprise parameters 

• Parameter model 


E. Enterprise 
Parameter 
Normalization 


• Normalised 
enterprise 
parameters 


• Verified Enterprise 
outcomes (desired) 

• Normalised LO 
outcomes 


F . Outcome 

Optimization 


• Enterprise 
curricula 


• Normalised enterprise 
parameters 

• Normalised LO 
parameters 

• Enterprise curricula 


G. Curriculum 
Selection 


• Enterprise 
training/ 
learning plan 


• Enterprise 

training/ learning 
plan 


H. Monitoring 


• Monitoring 
report 

• New plan 
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Modules A and B are conveniently located in a 
mapping unit of the system manager. Here, processes A (LO 
Analysis) and B (LO Description Normalization) are 
undertaken for all LO ' s whose descriptions are to be 
5 loaded into the description database. The mapping unit, 
who will also generate and add to the data for an Outcomes 
Thesaurus 13, undertakes this activity manually, or with 
computer assistance. 

Learning Objects 10 from a multiple suppliers, 
10 including the enterprise for which the training/learning 
program is being designed, are described in a normalised 
way using the processes A and B specified in Table 1. This 
takes place in the LO Analysis Module A and LO Description 
Normalization Module B shown in Fig. 1. 

15 Module A provides normalised descriptions, 

referred to as "elements of normalised LO description" 11 
which are supplied to module B which stores a Parameter 
Model database. The module B also receives an LO Supplier 
Description 12 and data from the Outcomes Thesaurus 13, 

20 and provides two outputs: normalised LO parameters 14 and 
normalised LO outcomes 15, which provide input to modules 
F and G, as described below. 

In module A, LO supplier descriptions are examined 
to see if they contain all the necessary elements for 

25 normalizing, using a Description Model. If they do, the 
normalised elements of LO description are taken from the 
supplier description and entered into a normalised format 
as required by module A. If they do not, the LO*s 10 are 
examined, the necessary elements to normalise the 

30 descriptions are identified and corresponding entries made 
into module A's normalised format. 

In module B, elements of LO supplier descriptions 
12 and, if necessary, elements of standard LO description 
11 from module A, are normalised using a Description 
35 Model, data from the Outcomes Thesaurus 13 and the 
Parameter Model database stored in module B. The 
descriptions established in module B are loaded into a 
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description database of module B for output as normalised 
LO parameters 14 and normalised LO outcomes 15. 

Similarly, outcomes and parameters are also 
described in a normalised way, using processes C and E 
5 respectively in an Enterprise Outcome Standardisation 
Module C and an Enterprise Parameter Normalization Module 
D. 

In module C, Enterprise Outcomes 16 are 
systematically compared one by one with normalised outcome 

10 names, synonyms and descriptions contained in the Outcomes 
Thesaurus 13. The Enterprise Outcomes 16 represent 
outcomes of any given training program, as defined by the 
enterprise, conforming to whatever model the enterprise 
uses, e.g. competency model. The level at which the 

15 outcome is expected, and the environment within which it 
is exhibited, are also taken into account. Equivalences 
are established, and each enterprise outcome is mapped to 
an outcome name or synonym in the Thesaurus. This activity 
is undertaken by someone familiar with the enterprise 

20 outcomes 16 using a programmed normalization module. 

The enterprise will also have the option of using 
a standard curriculum from a collection of Standard 
Curricula 17, in which case the normalization will have 
already been completed, or of modifying a standard 
25 curriculum, in which case only the modifications will be 
normalised. 

Module C provides normalised enterprise outcomes 
which are verified in the outcome verification module D. 
Each mapping is verified for consistency and coherence - 

30 exceptions are identified automatically, and a specialist 
decides how they will be treated. In the case where an 
enterprise outcome has not been successfully mapped, a new 
outcome or synonym is manually created. From these 
manually-adjusted inputs, a set of verified enterprise 

35 outcomes 18 is created automatically, using normalised 
language, for each training/ learning program, the new 
verified outcomes 18 being sent to the outcomes thesaurus 
13 for addition thereto. 
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These normalised/verified enterprise outcomes 18 
and normalised LO parameters 14 are then matched or 
"optimized" by process F to produce a choice of 
training/ learning programs (each constituting a 
5 "curriculum") for a group of students, viz. Enterprise 
Curricula 19. These curricula 19 are then ranked by 
process G in Curriculum Selection Module G according to 
enterprise parameters 21 to produce a single chosen 
curriculum, the enterprise training/ learning plan 20. 

In outcome optimization module F, using both the 
verified enterprise outcomes 18 and the LO outcomes 
(normalised LO parameters 14), a set of alternate 
curricula 19 are established, with a specialist from the 
enterprise (the "curriculum manager") making choices on 
how best to achieve the optimization, the computer system 
carrying out these choices. 

An enterprise parameter normalization module E is 
provided which inputs enterprise parameters 21 and outputs 
normalised enterprise parameters. The enterprise 
20 parameters 21 are provided by the enterprise, indicating 
which values it gives for each training/ learning program 
to the standard parameters contained in the Parameter 
Model. These are verified automatically. 

Curriculum selection takes place in module G, 
which inputs normalised enterprise parameters from module 
E and normalised LO outcomes 15 from module B. In module 
G, chosen curricula 19 from module 18 are ranked according 
to, firstly, the values given to the standard parameters 
and secondly, the weightings given to each parameter. The 
computer system presents to the curriculum manager a 
description of each of the outcomes and each of the LO's 
being optimized, selectable by the curriculum manager 
individually and in combinations, and provides the 
curriculum manager inter alia with an indicia of the match 
to requirements of a given curriculum through interactive 
indices, for example through these five interactive 
indices: 

1) percentage of enterprise outcomes which are 
covered by selected LO's; 
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2) percentage of LO outcomes which are in excess 
of enterprise outcomes; 

3) desired cost versus actual cost; 

4) desired duration versus actual duration; and 

5 5) desired mix of training/ learning methodologies 

versus actual mix. 

These indices change as the curriculum manager 
adds and subtracts LO's from the normalised LO outcomes 15 
and adds and subtracts enterprise outcomes from 18. At the 
10 end of this process, the curriculum manager chooses a set 
of curricula 19. 

The curriculum manager can thus interactively 
modify both the values and the weightings of relative 
importance of the parameters, and have the results 
15 presented to him on a computer screen or in a printed 
report. Parameters which can be expressed in relative 
terms (cost, duration, mix of training/learning 
methodologies) are presented interactively on the screen. 
When the curriculum manager decides that the optimum has 
20 been reached, a plan is automatically produced, on screen 
and/or by print-out. To the extent to which schedules and 
locations have been selected as parameters, the plan 
includes scheduling information. 

The implementation of the training program needs 
to be managed and followed-up to ensure success. This 
phase comprises mainly: curriculum assignment to the 
target audience; orientation of participants; monitoring 
and support of learners activities; planning of 
instructor-led sessions and participants enrollment; and 
follow-up costs. Principally, outcomes need to be measured 
and compared with the original objectives, to measure the 
efficiency of the program. This supposes the ability to 
measure all costs by category and compare to initial 
budget, and measure outcomes: did the program reach the 
goals? 

For this, the plan 20 is monitored in an optional 
module H according to the parameters chosen, periodic 
reports are produced, and optionally the plan is changed. 
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Using module H, the enterprise can regularly update the 
parameters selected as the plan is implemented. These are 
then compared with the parameter values and weightings 
originally chosen. Periodic reports (such as actual 
5 parameters vs. chosen parameters) are produced, a 
comparative report is automatically generated and if 
needed a new plan is produced or the existing one amended. 

Processes C to H are undertaken for each 
enterprise training/learning program. In many cases, an 
10 enterprise having undertaken C and D for one program will 
be able to reuse the generated outcomes 18 for other 
programs . 

The system according to the invention is 
advantageously delivered through an Application Service 

15 Provider (ASP) model via the internet. It can be part of a 
fully-scalable integrated management-centric B2B solution 
that involves both aggregation of on-line courses and 
Straight-Through Processing (STP) of training management 
functions. The system can thus be associated with 

20 management tools for sourcing, planning and budgeting the 
training needs, as well as enterprise training 
administration and student progress tracking, providing a 
fully-fledged collaborative e-learning environment for 
students and a broad spectrum of off-the-shelf content in 

25 all knowledge domains (IT, ERP and industry applications, 
skills and management etc.). Accessed through a tailored 
enterprise portal, such service will allow integration of 
enterprise-specific content and non-electronic courses. 

In a preferred embodiment, the system content is 
30 defined in a Content Description Language (CDL) which 
allows the definition of company or personal objectives in 
terms of outcomes. Outcomes are used to normalise the 
identification of the result that a given training module 
is expected to achieve. CDL therefore allows to define 
35 objectives in generic terms not immediately linked to a 
given set of training content, to compare alternate 
training solutions that are expected to achieve similar 
results and choose the most appropriate with regard to 
other characteristics (cost, delivery method, etc.), to 
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identify and present the global expected achievement of a 
set of training solutions and compare with the desired 
result, and to identify the additional training value of a 
set of training solutions with regard to the planned 
5 objectives. 

Managers and students will be able to construct 
their curriculum through a mixed selection of training 
solution and identification of desired outcomes. Using the 
system of the invention, they can optimize the training 
10 program or curriculum to match their objectives taking 
into account other constraints and objectives. 

Fig. 2 is a matrix illustrating learning outcomes 
as they apply to different learning objects. A learning 
object (LO) is a solution that enables a student , to 
15 acquire knowledge and skill in order to reach an outcome. 
A learning object can be of several types : lecture lab, 
computer-based training, book, test, etc. 

Each LO has attributes, including for instance: 
approval needed from the curriculum manager (yes/no) , if 
yes time range for approval, expressed in days with value 
from 0 to n; approval needed from the training 
administrator (yes/no), if yes time range for approval, 
expressed in days; last cancellation date (expressed in 
days before the LO start date); feedbacks related to the 
LO; pre-requisites - a set of outcomes that the student 
needs to own before registering for the LO; availability 
dates; for a lecture lab, the maximum number of people, 
the required number of computers and period, the required 
room and period, and the trainer /day required. 

The purpose of the curriculum builder is to select 
a set of LO that matches best a set of outcomes, Curricula 
are built by the curriculum manager, using the system 
according to the invention, in a global form for groups of 
students, and by the students themselves. To build a 
curriculum, the users can select LO's from the catalog of 
normalised LO parameters 14, select outcomes from the 
thesaurus 13 (viz. at the outcome optimization module F) 
and/or select standard curricula 17 offered by the system, 
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and combine and modify these in the curriculum selection 
module G. 

A "good" curriculum must present a high coverage 
for the selected outcomes; a low coverage of outcomes not 
5 selected and a low redundancy in outcomes coverage. This 
can be achieved manually in the curriculum selection 
module G. An automatic generation of a curriculum based on 
a selection of LO ' s and outcomes is also possible. The 
curriculum selection module G is designed to present 

10 information allowing the user to make intelligent choices, 
for instance: by being informed of the number of LO that 
address the same outcome, by being informed of the number 
of outcomes that are part and above the selected set for a 
given LO, by providing an overview of the coverage and 

15 redundancy of the entire curriculum, and by presenting a 
list of choices of LO addressing a selected outcome and 
the impact on the entire curriculum. 

A matrix such as that shown in Fig. 2 allows the 
user a good perception of the quality of the curriculum 

20 being built. References 30, 31, 32 indicate selected 
learning objects and references 40, 41, 42 indicate 
selected outcomes. As indicated in Fig. 2, the number of 
outcomes by selected LO is 1-within and 2-above for LO 30; 
1 -within and 1- above for LO 31; and also 1-within and 1- 

25 above for LO 32, whereas the number of LO's per selected 
outcome is 2 for outcome 40, 0 for outcome 41 and 1 for 
outcome 42 . 

The business objectives of an entire training 
program are set by the training manager. There are 

30 essentially three scenarios that drive the preparation of 
a training program, the competency plan, discrete explicit 
needs and business objectives. In the competency plan, the 
purpose of the program is to enhance specific competences 
within the organization, for example to develop security 

35 awareness. The discrete explicit needs approach means that 
the content of the training plan program is predefined, 
for example migration program to a new platform. In 
business objectives, the purpose of the training program 
is to solve a business problem, for example to reduce help 
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desk response time. As such, the objectives of the 
training program are expressed in free text associated to 
the training program. 

The invention will be further described by way of 
5 an example illustrating normalised LO parameters. Table 2 
shows an extract of an LO description, including elements 
of the Parameter description and the Outcome description. 
In this example, the LO is an electronic course teaching 
the use of Lotus™ Notes, supplied by Educational 
10 Multimedia Corporation, and the system manager is 
identified as TrainingFox. 

In Table 2, the Parameters are taken from the 
supplier's description or from the LO itself and inserted 
into the description. The Outcomes, however, are 
15 transformed. This occurs as indicated after the Table. 
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Table 2 

TrainingFox Learning Object Description 



Learning Object Identification and Use 



Title: | Lotus Notes R5 - Part 1 


Version: 


n/1 


3 1 Date of Version: 


n/k | Status: | Complete 


Provider: 




Educational Multimedia Corporation 


Provider's 


[D: 1 None obvious 


TrainingFox's ID: | Not assigned 


Pavment Required for Use: 


presumed Restricted Use: | presumed 


Managed Bv: i Educational Corporation 


Type of Learning Object: 


Course 


Learning Object Medium: 


rnrcoM 


M^ta-Data-Scheme Comoliance: 1 None known 


Location: 


TrainingFox Delivery Server 


Reviews: 


Nnne known 


Language: 


American English 


Size: 181MB 



T/C 


Taxonomy or Catalog 


Entries 




No Separate Taxonomy 


Taxonomy defined by Course Modules 

































Prerequisites: 


None stated formally - presumably basic PC operation 


Tested by (internal): 


Pre-assessment test, questions, and section tests 


Tested by (external): 


None known 



Coverage: 


Anticipated future Lotus 
Notes users 


Difficulty: 2 


Duration: 


Not relevant 


Semantic-Density: 


2 


Interactivity-Level: 


1 


Interactivity-Type: 


Mixed 


Typical-Age-Range: 


None 


Typical-Learning-Time: | 1 day 


Learning-Context(s): 


Vocational Training 


Learning Object User Role(s): To become Lotus Notes R5 user 





Keywords Provided by Provider 



None obvious 
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Usage Conditions and Required Support Resources 



Learning-Outcomes asserted by Provider (Top Ten) 



No 


Learning Outcome 


Type 


Level 


1 


Know key components of Lotus Notes R5 


Knowledge 


Basic 


2 


Know basic functions of Lotus Notes R5 


Knowledge 


Basic 


3 


Perform basic functions of Lotus Notes R5 


Skill 


Basic 


4 








5 








6 








7 








8 








9 








10 









Learning Object Analysis 



Analyst 


Role 


Date 


Version of 
Description 


Levels of 
Analysis 


John Smith- Jones 


Independent Analyst 


15-Nov-2000 


tLO 


Course 



Analyst's Assertions of Competences 



Competence 


Type 


Level 


Known basic concepts of Lotus Notes 


Knowledge 


Basic 


Elementary operations with Lotus Notes 
Databases 


Skill 


Basic 


Elementary operations with Lotus Notes Mail 


Skill 


Basic 


Elementary operations with Lotus Notes 
Calendar 


Skill 


Basic 


Elementary operations with Lotus Notes 
Address Book 


Skill 


Basic 


Elementary operations using Lotus Notes to 
access the World Wide Web 


Skill 


Basic 
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Transformation of outcomes takes place as follows: 

1. A client will establish a desired outcome for 
example using the following words: "Newly 
recruited employees will all have mastered those 

5 features of Lotus Notes R5 most frequently used, 

after having received training". 

2. The supplier has described the LO as follows: "At 
the end of this course, students will" be familiar 
with the most important elements of Lotus Notes 

10 R5, will understand the functionality, and will be 

able to undertake elementary operations". 

3. The Thesaurus will have, inter alia, the following 
entries : 

basic: synonyms = important, frequently... 
15 component: synonyms = element, . . . 

course: synonyms = training, . . . 

function: synonyms = feature, functionality, 
operation, . . . 

know: synonyms = master, understand, familiar, . . . 
20 perform: synonyms = undertake, able, . . . 

4. The Thesaurus is used to normalise the supplier's 
Outcome description as indicated under the caption 
"Learning-outcomes asserted by Provider (Top Ten) " 
in Table 2: 

25 "Know key components of Lotus Notes R5 

Know basic functions of Lotus Notes R5 
Perform basic functions of Lotus Notes R5". 

5. The Thesaurus is also used to link the clients 
desired output to the normalised description. The 

30 system will then present the course to the client 

as a candidate for inclusion in a client 
curriculum, as described previously. 

Many modifications may be made to the system and 
the method according to the invention within the scope of 
35 the appended claims. 



